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FOREWORD 

This  research  and  development  was  performed  in  support  of  Advanced  Develop- 
ment Subproject  ZPN01.06,  Advanced  Navy  Recruiting  System. 

The  study  focused  on  the  development  and  evaluation  of  interest  scales 
to  aid  midshipmen  in  vocational-educational  guidance.  More  specifically, 
empirical  scales  on  the  Strong  Vocational  Interest  Blank  were  developed  to 
determine  the  degree  of  similarity  between  a midshipman's  measured  interests 
and  those  of  graduates  in  each  of  the  three  academic  majors:  (1)  Engineering- 

Weapons,  (2)  Mathematics-Science,  and  (3)  Humanities-Social  Sciences.  This 
information  would  be  useful  in  the  selection  of  an  academic  major. 

The  assistance  of  the  Naval  Academy  throughout  all  phases  of  this  research 
is  gratefully  acknowledged.  The  considerable  efforts  of  LCDR  Robert  D.  McCullah, 
formerly  with  the  Department  of  Behavioral  Science,  were  responsible  for 
expediting  this  investigation  and  helped  ensure  its  successful  completion.  The 
interest,  cooperation,  and  support  of  RADM  Robert  W.  McNitt,  USN  (Ret.), 

Dean  of  Admissions;  Dr.  John  F.  Kelley,  Jr.,  Assistant  Dean  for  Academic 
Affairs;  and  Dr.  Gregory  Mann,  Chairman,  Behavioral  Science  Department,  are 
especially  appreciated. 


J.  J.  CLARKIN 
Commanding  Officer 
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Probl em 

Effective  procurement,  development,  and  retention  of  officer  person- 
nel continue  to  be  important  objectives  for  the  Navy.  The  U.S.  Naval 
Academy,  one  important  source  of  commissioned  officers,  offers  education 
organized  under  three  broad  academic  areas:  (1)  Group  I — Engineering- 

Weapons,  (2)  Group  II — Mathematics-Science,  and  (3)  Group  III — Humanities- 
Social  Science.  Although  midshipmen  are  given  choices  among  these  program 
areas,  Academy  policy  requires  that  certain  proportions  of  each  class  select 
majors  in  each  area.  The  general  problem  addressed  in  the  present  research 
is  to  provide  midshipmen  with  information  that  will  help  them  to  choose 
academic  majors  that  are  personally  satisfying  while,  at  the  same  time, 
ensuring  that  Academy  goals  are  achieved. 

Objective 

This  study  developed  and  evaluated  interest  scales  designed  to  provide 
an  index  of  similarity  between  a person's  measured  interests  and  those  of 
graduates  in  each  of  the  three  broad  academic  areas. 

Approach 


Strong  Vocational  Interest  Blank  (SVIB)  responses  and  ultimate  choice 
of  major  field  were  available  for  the  Classes  of  1971,  1972,  1973,  and  1975. 
Four  separate  interest  scales  were  developed  on  the  Class  of  1973:  (1)  the 

Engineering-Weapons  (E-W)  Scale,  designed  to  separate  Group  I majors  from 
others  (II  & III),  (2)  the  Mathematics-Science  (M-S)  Scale,  designed  to 
separate  Group  II  majors  from  others  (I  & III),  (3)  the  Humanities-Social 
Science  (H-S)  Scale,  designed  to  separate  Group  III  majors  from  all  others 
(I  & II),  and  (4)  the  Applied-Science  (A-S)  Scale,  designed  to  separate 
Group  I majors  from  Group  II  majors. 

The  four  individual  scales  were  cross-validated  separately  for  the 
Classes  of  1971,  1972,  and  1975.  Four  alternative  strategies  for  utilizing 
the  scales  were  compared  with  chance  results  and  with  each  other  in  terms  of 
classification  accuracy. 

Cross-validation  of  the  individual  scales  produced  the  following  ranges 
of  biserial  correlations:  (1)  the  E-W  Scale  - from  .42  to  .50,  (2)  the  M-S 

Scale  - from  .16  to  .37,  (3)  the  H-S  Scale  - from  .66  to  .73,  and  (4)  the 
A-S  Scale  - from  .27  to  .32.  All  biserial  validities  for  each  scale  in  each 
sample  were  highly  significant  (p  < .01). 

The  effectiveness  of  four  alternative  strategies  for  utilizing  the  informa- 
tion provided  by  the  interest  scales  was  examined.  The  "hit  rate"  (percent 
correct  classifications)  was  employed  as  the  basis  for  evaluating  the 
strategies.  All  four  strategies  were  found  to  significantly  improve  upon 
chance  result  (p  < .01). 
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Conclusions 

Consideration  of  institutional  goals  in  conjunction  with  classification 
accuracy  lead  to  the  conclusion  that,  of  the  four  strategies  for  using  the 
interest  scales,  the  best  was  an  optimal  personnel  assignment  algorithm 
with  current  quotas  for  the  three  academic  areas.  The  assignments  generated 
by  the  algorithm  would  provide  recommendations  on  choice  of  major  area. 
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INTRODUCTION 

Problem 

The  United  States  Naval  Academy  is  an  important  source  of  commissioned 
officers  that  offers  numerous  alternative  career  opportunities.  Until  1959, 
the  curriculum  was  uniform  and  consisted  of  a "lock-step"  sequence  of  courses. 
There  was  no  program  involving  different  majors  or  minors,  and  no  elective 
courses  were  available.  In  1959,  advanced  placement  based  upon  test  results 
was  introduced  for  the  new  plebe  class  (Class  of  1963),  and  an  electives 
program  was  initiated.  All  midshipmen  still  had  Naval  Science  as  an  academic 
major;  however,  when  they  had  completed  the  plebe  year,  those  who  were  most 
academically  qualified  were  offered  the  opportunity  of  participating  in  the 
electives  program;  that  is,  they  could  choose  a minor  in  one  of  twelve  sub- 
ject areas. 

The  "Majors  Program,"  instituted  in  1969,  provided  majors  organized  under 
four  broad  academic  areas:  (1)  Engineering,  (2)  Science,  (3)  U.S.  and  Inter- 

national Studies/English  and  History,  and  (4)  Management.  However,  beginning 
with  1973  (Class  of  1977),  curricula  for  these  areas  were  structured  into 
three  groups:  (1)  Group  I — Engineering-Weapons,  (2)  Group  II — Mathematics- 

Science,  and  (3)  Group  III — Humanities-Social  Science.  The  curricula  for 
the  management  majors  were  substantially  altered  and  grouped  administratively 
under  Group  II.  For  convenience,  in  the  remainder  of  this  report,  the  term 
"original"  will  designate  the  organization  of  majors  that  existed  prior  to 
and  for  the  Class  of  1976;  and  the  term  "current,"  to  the  organization  that 
began  with  the  Class  of  1977. 

As  indicated  in  The  Majors  Program  (1974),  published  by  the  Academy  as 
Academic  Dean  Notice  1531,  there  is  a marked  emphasis  on  the  technical  majors 
of  Groups  I and  II: 

The  Naval  Academy  policy  on  the  selection  of  majors  is  clear. 

Each  midshipman  selects  a major  which  will  meet  the  needs  of  the 
Navy  and  at  the  same  time  be  interesting  to  him.  The  needs  of 
the  Navy  take  first  priority  and  it  has  been  determined  that  80 
percent  of  the  Class  shall  take  a technical  major,  i.e..  Group 
I or  II,  and  20  percent  may  choose  Group  III.  Hopefully,  the 
selection  of  majors  by  the  Class  of  1978  will  meet  the  80/20 
quota.  If  the  desired  distribution  is  not  obtained  by  an  open, 
free  selection  process,  steps  will  be  taken  to  adjust  the  dis- 
tribution to  meet  the  Navy  goals.  (p.  9) 

The  Academy  was  directed  to  increase  the  proportion  of  engineering  and 
science  majors  from  65  percent  to  the  80  percent  referred  to  above  by  the 
Bureau  of  Naval  Personnel.  This  increase  is  necessary  if  the  Navy  is  to 
meet  present  and  future  requirements  for  high  quality  naval  officers  with 
engineering  and  science  backgrounds.  If  the  characteristics  of  incoming 
midshipmen  remain  the  same  (which  is  likely  to  be  the  case),  there  is  little 
reason  to  expect  that  15  percent  more  of  them  will  select  engineering  or 
science  majors.  Thi3  decision  point  is  reached  in  the  Spring  of  the  plebe 
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year,  which  is  earlier  than  in  civilian  colleges,  where  a choice  of  major 
is  most  often  made  at  the  conclusion  of  the  sophomore  year.  As  a result, 
early  vocational-educational  guidance  based  at  least  in  part  on  interests 
may  play  an  especially  important  role  in  the  Naval  Academy  setting. 


Background 

A substantial  amount  of  research  on  interests  and  their  relationship 
to  various  criteria  of  performance  and  satisfaction  has  been  accomplished 
in  the  past  10  to  15  years.  The  Strong  Vocational  Interest  Blank  (SVIB) 
has  been  found  to  be  one  of  the  most  useful  psychometric  instruments  for 
measuring  these  interests  (Campbell,  1966). 

Abrahams  and  Neumann  (1973)  studied  the  relationship  between  SVIB 
responses  and  performance  at  the  U.S.  Naval  Academy.  Specifically,  these 
investigators  developed  scales  for  predicting  three  criteria:  motivational 

disenrollment , academic  disenrollment , and  military  aptitude.  All  of  these 
new  scales  were  found  to  be  significantly  related  to  their  respective  criteria 
in  cross-validation  samples. 

In  another  study,  Neumann  and  Abrahams  (1974)  developed  an  SVIB  scale 
designed  to  identify  Naval  Academy  applicants  with  engineering  and  science 
interests.  The  Class  of  1973  was  split  into  key-development  and  cross- 
validation  samples.  The  criterion  employed  was  dichotomous:  Engineering- 

Science  majors  versus  "Other"  majors.  A biserial  validity  of  .57  was  obtained 
for  this  Engineering-Science  (E-S)  Scale  in  the  cross-validation  sample. 
Further,  application  of  the  E-S  scale  to  the  class  of  1976  yielded  a biserial 
validity  of  .62.  The  results  of  this  investigation  were  also  reported  in  a 
paper  presented  at  an  Air  Force  symposium  (Abrahams  & Neumann,  1974). 

The  E-S  Scale  was  developed  as  an  aid  in  selecting  students  with  engineer- 
ing or  science  interests  from  the  pool  of  Academy  applicants.  The  operational 
selection  composite  involves  a number  of  different  predictors.  Thus,  the 
relationship  between  the  E-S  Scale  and  the  current  predictors  was  examined  and 
the  validity  of  alternative  composites  was  evaluated  against  various  criteria 
(e.g.,  cumulative  grade  point  average,  major  choice,  etc.)  (Neumann  & Abrahams, 
1976).  As  a result  of  this  research,  both  the  Disenrollment  Scale  and  the 
Engineering-Science  Scale  will  be  employed  in  computing  the  candidate  multiple 
for  selecting  applicants  for  the  Class  of  1980. 

Purpose 


The  global  purpose  of  this  research  is  the  development  of  tools  that  will 
aid  the  Naval  Academy  staff  in  the  educational-vocational  guidance  of  the 
students.  The  specific  objective  of  this  effort  is  to  construct  interest  scales 
that  will  provide  an  index  of  the  degree  of  similarity  between  an  individual's 
Interests,  as  measured  by  the  Strong  Vocational  Interest  Blank,  and  those  of 
persons  who  have  successfully  graduated  in  each  of  the  three  current  broad 
academic  areas  at  the  Academy.  This  information  on  individual  interests  can 
be  used.  In  conjunction  with  institutional  requirements,  to  guide  midshipmen 
in  the  selection  of  an  academic  major. 
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PROCEDURE 


Instrument 


The  1966  edition  of  the  Strong  Vocational  Interest  Blank  (SVIB)  for  men 
is  a paper-and-pencil  inventory  consisting  of  399  items  relating  to  activities 
in  23  basic  interest  areas.  These  areas  include  occupational  specialties, 
school  subjects,  amusements  and  hobbies,  and  general  activities.  The 
respondent  is  asked  to  indicate  his  degree  of  interest  in  the  activity  described 
in  the  particular  item  by  responding  with  "Like,"  "Indifferent,"  or  "Dislike." 

In  1966,  1967,  and  1968,  this  version  of  the  SVIB  was  administered  to  all 
midshipmen  (selectees  for  the  Classes  of  1971,  1972,  and  1973)  on  an  experi- 
mental basis  at  the  inception  of  their  respective  plebe  summers.  In  1969, 
it  was  administered  on  a voluntary  basis  as  part  of  the  application  procedure. 
However,  beginning  in  1970,  for  selection  of  the  class  of  1975,  it  became  a 
required  part  of  the  application  packet. 

Sample 

Since  complete  SVIB  data  were  available  for  the  Classes  of  1971  through 
1973  and  1975,  these  classes  were  selected  for  construction  and  evaluation 
of  the  new  SVIB  scales.  As  indicated  above,  the  SVIB  was  administered  to 
all  selectees  for  the  Classes  of  1971  through  1976.  Those  persons  who  failed 
to  graduate  were  removed  from  these  three  samples.  Further,  since  the  manage- 
ment science  major  area  was  eliminated  in  1973,  for  the  Class  of  1977,  those 
persons  graduating  with  a management  major  also  were  removed  from  the  1971, 

1972,  and  1973  samples.  For  the  Class  of  1975,  the  SVIB  was  administered  to 
all  applicants.  However,  the  sample  included  only  the  graduates. 

The  number  and  percentage  of  persons  in  each  class  majoring  in  each  of 
the  three  current  broad  academic  areas  are  shown  in  Table  1.  As  shown,  two 
sets  of  data  are  provided  for  the  Class  of  1975.  In  the  first  set,  which 
portrays  the  "original"  organization  of  majors,  which  existed  prior  to  and 
for  the  Class  of  1976,  General  Engineering  majors  (N  = 50)  are  included  in 
Group  I and  Management-Science  majors  (N  = 77)  have  been  excluded.  In  the 
second  set,  which  groups  midshipmen  to  reflect  the  "current"  organization, 
which  began  with  the  Class  of  1977,  General  Engineering  and  Management  majors 
are  included  in  Group  II.  The  first  data  set  will  provide  a realistic  picture 
of  the  effectiveness  of  the  new  SVIB  scales  if  they  were  to  be  implemented 
under  the  original  organization  of  academic  majors;  and  the  second  data  set, 
the  current  organization. 

Criteria 


The  criterion  for  each  scale  developed  was  membership  versus  nonmember- 
ship in  the  respective  major  group.  Four  separate  problems  were  addressed: 
(1)  differentiation  of  Group  I from  Groups  II  and  III;  (2)  differentiation 
of  Group  II  from  Groups  I and  III;  (3)  differentiation  of  Group  III  from 
Groups  I and  II;  and  (4)  differentiation  of  Group  I from  Group  II. 
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Construction  of  Scales 


The  1973  sample  was  used  to  construct  the  following  four  scales  designed 
to  address  the  four  differentiation  problems  stated  above:  (1)  the  Engineer- 

ing-Weapons (E-W)  Scale,  (2)  the  Mathematics-Science  (M-S)  Scale,  (3)  the 
Humani t ies-Social  Science  (H-S)  Scale,  and  (A)  the  Applied  Science  (A-S) 

Scale . 

An  "empirical  criterion  keying"  approach  was  used  to  select  those  SVIB 
items  having  the  75  best  responses  for  each  of  the  four  problems  addressed. 
Previous  research  has  indicated  that  this  number  of  responses  produces 
effective  SVIB  scales.  The  proportions  of  high  and  low  criterion  group1 
members  who  endorsed  each  of  the  response  alternatives  for  each  of  the  items 
were  computed,  and  the  difference  between  these  two  endorsement  rates  was 
determined.  The  items  containing  the  75  responses  exhibiting  the  greatest 
differences  between  endorsement  rates  of  the  high  and  low  criterion  groups 
were  selected  for  subsequent  analysis. 

Twenty  alternative  item  response  weighting  procedures  were  examined 
(Sands,  1975).  These  procedures  ranged  in  complexity  from  a simple  unit- 
weighting  method  to  more  involved  methods,  such  as  using  Bayes'  Theorem  to 
obtain  a response  weight  reflecting  the  posterior  probability  of  being  a high 
criterion  group  member,  given  that  the  response  alternative  was  endorsed. 

The  general  conclusion  reached  was  that  the  more  esoteric  methods  of  dif- 
ferentia wt  ;hting  offered  no  practical  advantage  over  the  simple-unit  weight- 
ing procedure. 

The  response  weighting  method  adopted  for  the  four  scales  described  in 
this  report  was  a dimensional ized  version  of  unit  weighting,  as  recommended 
by  Campbell  (1971).  This  procedure  considers  each  individual  item  as  a 
continuum  ranging  from  "Like"  at  one  end  to  "Dislike"  at  the  other.  If  one 
end  of  the  continuum  received  a unit  weight  based  upon  the  absolute  difference 
in  endorsement  rates  between  the  high  and  low  criterion  groups,  a unit  weight 
affixed  with  the  opposite  sign  is  given  to  the  other  end.  If  "Indifferent" 
is  the  only  response  alternative  that  exhibited  the  necessary  difference 
between  the  endorsement  rates  of  the  high  and  low  criterion  groups,  the  item 
obviously  does  not  have  the  assumed  underlying  continuum  and,  therefore,  was 
eliminated  from  the  scale.  Campbell  (1971)  reports  that  dimensionalizing 
does  not  affect  the  validity  of  a scale  but  does  provide  slight  improvement 
in  test-retest  reliability. 

Development  of  Prediction  Equations 

Four  separate  linear  regression  equations  were  constructed  on  the  1972 
sample,  one  for  each  of  the  four  differentiation  problems  addressed.  By 
coding  group  membership  vs.  nonmembership  in  a binary  fashion,  the  simple 
regression  equations  estimated  the  probability  of  high  criterion  group  member- 
ship as  a linear  function  of  the  raw  score  on  the  appropriate  scale. 


1 The  term  "high  criterion  group"  is  used  throughout  the  paper  to 
indicate  the  group  for  whom  the  scale  is  being  developed;  and  "low  criterion 
group,"  to  indicate  nonmembership  in  that  group. 
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Evaluat ion 


Individual  Scales 


The  effectiveness  of  the  four  individual  scales  was  evaluated  in  a 
number  of  ways.  The  biserial  validities  and  percent  overlap  indexes 
(Tilton,  1937)  were  computed  separately  for  each  scale  in  each  sample. 

Also,  an  expectancy  table  which  shows  the  proportion  of  high  criterion 
group  members  scoring  in  each  quintile  of  the  appropriate  raw  score  pre- 
dictor distribution,  was  constructed  for  each  scale  in  each  sample. 

Classification  Strategies 

Four  alternative  classification  strategies  were  evaluated.  The 
first  strategy  examined  the  separate  probability  estimates  of  membership 
in  the  three  academic  areas.  An  individual  was  classified  into  the  group 
for  which  his  score  indicated  the  highest  probability.  Previous  research 
(Sands  & McCullah,  1974)  demonstrated  that  separating  Group  I members  from 
Group  II  members  was  more  difficult  than  isolating  Group  III  persons  from 
Groups  I and  II.  The  second  classification  strategy  was  designed  with  this 
finding  in  mind.  Initially,  the  probability  of  Group  III  membership  was 
estimated  using  a person's  raw  score  on  the  H-S  scale.  If  this  probability 
was  greater  than,  or  equal  to,  0.5,  he  was  classified  as  a Group  III  member; 
otherwise,  he  was  considered  as  either  a Group  I or  Group  II  member.  Using 
his  raw  score  on  the  A-S  Scale  (designed  to  differentiate  between  Groups  I 
and  II),  his  probability  of  Group  I membership  was  estimated.  If  this 
estimate  was  greater  than,  or  equal  to  0.5,  he  was  classified  as  a Group  I 
member;  otherwise,  he  was  predicted  to  be  a Group  II  member.  It  should  be 
noted  that  neither  of  the  first  two  strategies  considers  institutional  goals 
for  allocation  to  the  three  academic  groups. 

The  third  and  fourth  strategies  employed  a personnel  assignment 
algorithm,7  which  optimally  assigned  each  person  to  one  of  three  major  areas 
while  meeting  quotas  for  each  area.  The  payoff  matrix  for  both  strategies 
employed  the  three  probabilities  of  group  membership  estimates  as  the  payoffs 
of  assigning  the  individual  to  each  of  the  three  major  groups.  The  third 
strategy  used  quotas  set  equal  to  the  number  of  persons  who  actually  majored 
in  each  academic  area;  and  the  fourth,  quotas  of  40  percent,  40  percent,  and 
20  percent  for  Groups  I,  II,  and  III  respectively.  As  previously  indicated, 
the  Academy  has  published  a goal  of  80  percent  for  technical  majors  (F.ngineer- 
ing-Weapons  and  Mathematics-Science  majors)  and  20  percent  for  nontechnical 
majors  (Humanlties-Social  Science  majors).  The  desired  allocation  between 
Engineering-Weapons  (Group  I)  majors  and  Mathematics-Science  (Group  II) 
majors  has  not  been  specified.  To  estimate  these  separate  quotas,  the  Glass 
of  1978  was  examined.  The  observed  distribution  of  majors  for  Groups  I,  II, 
and  III  was  41,  39,  and  20  percent  respectively.  Therefore,  for  the  purpose 
of  illustration,  the  quotas  for  the  first  two  major  groups  were  assumed  to 
be  equal. 


7The  algorithm  employed  was  the  Ford-Fulkerson  routine  (1956)  for  solving 
the  Hitchcock-Koopmans  transportation  problem.  The  computer  program  used  was 
developed  by  Wolfe  (1964). 
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The  results  for  each  strategy  were  evaluated  separately  for  each 
sample.  A three-by-three  matrix  with  actual  group  membership  on  one 
dimension  and  predicted  (or  "assigned")  group  membership  on  the  other 
dimension  portrayed  the  obtained  results.  The  frequencies  on  the  diagonal 
were  correct  classifications  and  the  proportion  of  correct  classifications, 
or  "hit  rate,"  was  used  as  an  index  of  strategy  effectiveness. 

The  obtained  results  for  each  strategy  in  each  sample  were  compared 
with  chance  level  results.  Finally,  the  four  alternative  strategies  were 
compared. 
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RESULTS 


Evaluation  of  Individual  Scales 


Engineering-Weapons  (E-W)  Scale. 


Table  2 presents  statistical  results  for  the  E-W  Scale  for  each 
Academy  class.  As  shown,  cross-validation  of  this  scale  yielded  biserial 
correlations  ranging  from  .42  to  .50,  all  of  which  are  highly  significant 

(p  < .01). 

The  percent  overlap  is  another  index  that  is  useful  in  describing 
the  extent  to  which  a predictor  can  separate  two  groups  (e.g.,  Engineering- 
Weapons  majors  from  all  others).  This  index  is  based  upon  the  difference 
in  means  of  the  two  groups  divided  by  the  average  of  the  two  standard 
deviations  (Tilton,  1937).  It  can  vary  from  zero,  representing  perfect 
separation  of  the  score  distributions  of  the  two  groups,  to  100,  represent- 
ing a complete  overlap.  Hence,  a low  percent  overlap  indicates  a highly 
effective  predictor  variable.  As  shown  in  Table  2 the  cross-validated  per- 
cent overlap  coefficients  for  the  E-W  Scale  range  from  65  to  70.  The  magni- 
tude of  these  index  indicates  relatively  good  separation  of  criterion  groups. 

The  practical  significance  of  applying  the  E-W  Scale  can  be  illustrated 
using  expectancy  charts,  as  shown  in  Figure  1.  The  raw  score  categories  for 
all  expectancy  charts  were  formed  by  combining  the  raw  score  distributions  for 
the  classes  of  1971  and  1972.  This  total  distribution  was  divided  into  quin- 
tiles, each  fifth  as  nearly  equal  in  sample  size  as  possible.  Then,  treating 
each  class  separately,  the  percentage  of  Engineering-Weapons  majors  was 
computed  for  each  quintile.  As  indicated  in  Figure  la,  those  persons  in 
the  Class  of  1971  scoring  in  the  top  quintile  of  the  E-W  Scale  selected 
Engineering-Weapons  majors  almost  six  times  more  frequently  than  those  persons 
scoring  in  the  lowest  quintile.  Of  the  persons  scoring  in  the  top  60  percent 
of  the  E-W  score  distribution,  46  percent  were  Engineering-Weapons  majors,  as 
compared  with  a base  rate3  of  34  percent. 

Figures  lb,  lc,  and  Id  portray  the  effectiveness  of  the  E-W  Scale  as 
applied  to  the  remaining  three  cross-validation  problems.  The  results  are 
similar  to  those  obtained  for  the  Class  of  1971. 

Mathematics-Science  (M-S)  Scale 

The  results  of  statistical  analyses  on  the  M-S  Scale  are  presented  in 
Table  3.  The  cross-validated  biserial  validity  coefficients  range  from  .16 
to  .37,  all  of  which  are  highly  significant  (p  < .01).  These  validities  cor- 
respond to  overlap  coefficients  ranging  from  89  to  76.  These  high  overlap 
coefficients  reflect  the  difficulty  of  differentiating  between  the  Mathematics- 
Science  majors  and  persons  majoring  in  the  other  two  broad  academic  areas. 


3The  base  rate  is  the  proportion  of  high  criterion  group  members  in  the 
total  distribution.  For  example,  in  the  Class  of  1971,  there  were  225  Engineer- 
ing-Weapons majors  out  of  a total  of  660,  yielding  a base  rate  of  .34. 
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Scale  and  choice  of  major  field  for  the  Naval  Academy  classes. 
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Figure  1.  (Continued) 
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Figure  1.  (Continued). 


Descriptive  Statistics  for  the  Mathematics-Science  (M-S)  Scale 


These  results  are  based  upon  the  current  majors  organization. 
♦Statistically  significant  (p  < .01),  based  upon  Alf  & Abrahams  (1971). 


Figure  2 illustrates  the  lack  of  sharp  differentiation  between 
Mathematics-Science  majors  and  others.  For  the  Class  of  1971,  if  only 
those  persons  scoring  in  the  top  quintile  of  the  M-S  Scale  distribution 
are  considered,  only  47  percent  were  Mathematics-Science  majors,  which  is 
not  markedly  higher  than  the  40  percent  base  rate. 

Humanities-Social  Science  (H-S)  Scale 


Table  4 presents  the  results  of  a statistical  evaluation  of  the  H-S 
Scale.  The  biserial  validity  coefficients  range  from  .66  to  .73  in  the  cross- 
validation  samples,  all  of  which  are  highly  significant  (p  < .01).  The  per- 
cent overlap  coefficients  corresponding  to  these  validities  range  from  53  to 
46. 


Figure  3 illustrates  the  high  effectiveness  of  the  H-S  Scale.  Of  all 
the  persons  in  the  Class  of  1971  scoring  in  the  top  quintile  of  the  H-S 
Scale  score  distribution,  67  percent  were  majoring  in  the  Humanities- 
Social  Science  area,  as  contrasted  with  only  1 percent  of  those  scoring  in 
the  bottom  quintile.  The  base  rate  for  the  Class  of  1971  was  26  percent. 

Applied  Science  (A-S)  Scale 

Results  of  statistical  analyses  on  the  A-S  Scale  are  shown  in  Table  5. 
Cross-validated  biserial  correlations  ranged  from  .27  to  .32  and,  statistically, 
are  highly  significant  (p  < .01).  The  corresponding  percent  overlap  coefficients 
ranged  from  32  to  79. 

Figure  4 illustrates  the  effectiveness  of  the  A-S  Scale  for  the  four 
cross-validation  problems.  In  the  Class  of  1971,  over  twice  as  many  Engineer- 
ing-Weapons majors  scored  in  the  top  quintile  of  the  A-S  Scale  distribution 
ns  scored  in  the  bottom  quintile  of  the  score  distribution. 

Evaluation  of  Classification  Strategies 


The  results  for  the  four  alternative  classification  strategies  wore 
evaluated  separately  for  each  sample.  A three-by-three  matrix,  with  actual 
group  membership  on  one  dimension  and  predicted  or  assigned  group  membership 
on  the  other  dimension,  was  constructed  for  each  strategy,  as  applied  to  each 
samp  1 e . 


A corresponding  matrix  representing  chance  results  was  constructed  for 
each  strategy  in  each  sample.  These  detailed  results  are  presented  in  the 
Appendix.  The  hit  rate  obtained  for  each  strategy  was  compared  with  the  hit 
rate  that  would  have  been  obtained  by  chance.  The  results  for  the  four 
alternative  strategies  are  presented  in  Table  6.  The  obtained  results  were 
significantly  better  (p  < .01)  than  the  corresponding  chance  results  for  every 
strategy  examined  for  each  separate  sample. 

As  previously  Indicated,  the  first  two  strategies  are  based  solely  upon 
probability  of  group  membership  estimates  and  institutional  requirements  are 
ignored.  In  every  sample,  the  classification  accuracy  of  Strategy  //2  equals 
or  exceeds  the  accuracy  of  Strategy  #1.  This  finding  was  expected,  as  the 
second  strategy  employs  more  information  than  is  used  by  the  first  strategy. 
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Percentage  of  Persons  Expected  to  Select 


Figure  2.  Expectancy  charts  showing  the  relationship  between  the  M-S 

Scale  and  choice  of  major  field  for  the  Naval  Academy  classes. 
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Figure  2.  (Continued) 


Lgure  2.  (Continued) 


igure  2.  (Continued) 


Descriptive  Statistics  for  the  Humanities-Social  Science  (H-S)  Scale 
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"These  results  are  based  upon  the  current  majors  organization. 
*Statistically  significant  (p  < .01),  based  upon  Alf  & Abrahams  (1971). 
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Figure  3.  Expectancy  charts  showing  the  relationship  between  the  H-S 

Scale  and  choice  of  major  field  for  the  Naval  Academy  classes. 
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Percentage  of  Persons  Expected  to  Select 
Humanities-Social  Science  Major  in  Each 


Descriptive  Statistics  for  the  Applied  Science  (A-S)  Scale 
for  the  Naval  Academy  Classes 


These  results  are  based  upon  the  current  majors  organization. 
*Statistically  significant  (p  < .01),  based  upon  Alf  & Abrahams  (1971). 


Figure  4.  Expectancy  charts  showing  the  relationship  between  the  A-S 

Scale  and  choice  of  major  field  for  the  Naval  Academy  classes. 
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Figure  4.  (Continued) 


Class  of  1975 — Current  najor  groups. 
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Table  6 

i 

Comparison  of  Obtained  and  Chance  Results  for 


Alternat ive 

Classif icat ion 

Strategies 

Naval 

Academy 

Class 

N 

Obtained 

Results 

Chance 

Resul t s 

Improvement 
Over  Chance' 

Humber 

lilts 

Hit 

Rate 

X 

Number 

Hits 

Hit 

Rate 

X 

Number 

Hits 

Percent 

Improvement 

Strategy  It  1 

1971 

660 

320 

48.5 

2 39 

36.2 

81 

33.9 

1972 

646 

364 

56.3 

267 

41.4 

97 

36.  3 

1 97  Tb 

663 

388 

58.5 

271 

40.9 

117 

43.2 

1975c 

699 

326 

46.6 

233 

33.3 

93 

39.9 

1975d 

776 

426 

54.9 

328 

42.3 

98 

29.9 

Strategy  82 

1971 

660 

330 

50.0 

248 

37.6 

82 

33.1 

1972 

646 

364 

56.3 

279 

43.1 

35 

30.5 

1973b 

663 

401 

60.5 

279 

42.1 

122 

43.7 

1975C 

699 

331 

47.4 

24  2 

34.6 

39 

36.8 

1975d 

776 

432 

55.7 

338 

43.6 

94 

27.8 

Strategy  II 3 

1971 

660 

345 

52.3 

227 

34.4 

118 

52.0 

1972 

646 

365 

56.5 

235 

36.4 

130 

55.3 

1 9 7 1b 

663 

410 

61.8 

241 

36.3 

169 

70. 1 

1975C 

699 

367 

52.5 

24  3 

34.8 

124 

51.0 

1 9 7 5d 

776 

403 

51.9 

291 

37.5 

112 

38.5 

Strategy  04 

1971 

660 

342 

51.8 

230 

34.8 

112 

48.7 

1972 

646 

356 

51.1 

225 

34.8 

131 

58.2 

197  3b 

663 

393 

59.3 

2 30 

34.7 

163 

70.9 

1975C 

699 

366 

52.4 

247 

35.3 

119 

48.2 

1975d 

776 

389 

50. 1 

278 

35.8 

111 

39.9 

aObtained  results  are  significantly  more  accurate  than  chance  results  (p  < .01). 

bThe  statistics  for  the  Class  of  1971  are  Inflated,  as  the  scales  vere  constructed  usinc  these  persons. 
1 These  results  are  based  upon  the  original  majors  organization. 
dThese  results  are  based  upon  the  current  majors  organization. 
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The  third  and  fourth  strategies  employ  a personnel  assignment  algorithm 
which  maximizes  the  payoff  of  assignments  while  meeting  specified  quotas 
for  the  three  groups.  With  the  single  exception  of  the  Class  of  1975  (as 
majors  are  currently  organized),  Strategy  // 3 is  the  most  accurate  of  the 
four  alternative  strategies.  This  result  was  expected,  as  the  quotas  for 
the  three  groups  were  set  equal  to  the  obtained  historical  distribution  of 
majors  for  this  third  strategy.  In  effect,  this  procedure  assumes  that  the 
obtained  distribution  was  the  desired  distribution. 

In  Strategy  // 4,  the  quotas  were  set  equal  to  the  more  realistic  distribu- 
tion of  40,  40,  and  20  percent  for  the  three  groups.  As  indicated  in  Table  6, 
the  classification  accuracy  of  Strategy  #4  suffers  a slight  decrement  in 
comparison  to  Strategy  #3.  Again,  this  finding  was  expected,  as  the  simulated 
institutional  policy  regarding  quotas  constrained  the  solution. 
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DISCUSSION  AND  CONCLUSIONS 


Individual  Versus  Institutional  Decisions 


Cronbach  and  Gleser  (1965)  classified  decisions  as  either  individual 
or  institutional.  In  an  individual  decision,  the  preferred  course  of  action 
varies  from  one  person  to  another,  is  dependent  upon  the  value  system  of  the 
individual,  and  does  not  reflect  institutional  goals.  Strategies  If 1 and  If? 
may  be  considered  as  strategies  of  the  individual  type,  since  the  one  factor 
they  consider  in  making  classification  decisions  is  probability  of  group 
membership . 

Institutional  decisions  attempt  to  produce  a maximum  payoff  from  a series 
of  similar  decisions.  Such  decisions  may  consider  personal  interests  of 
individuals,  but  only  insofar  as  these  interests  impact  upon  the  institutional 
objectives.  Strategies  #3  and  If  A may  be  considered  as  strategies  of  the  in- 
stitutional type,  since  the  quotas  they  employ  for  the  three  major  academic 
groups  reflect  institutional  goals  rather  than  the  personal  goals  of  the 
students . 

Additional  Information  Requirements 

The  scope  of  this  investigation  was  limited  in  terms  of  both  predictor 
and  criterion  variables  examined . The  predictors  evaluated  were  empirically- 
derived  interest  scales.  Other  predictor  data  which  would  be  considered  in 
a realistic  tryout  would  certainly  include  information  on  aptitude  and  previous 
achievement . 

The  only  criterion  used  in  this  study  was  group  membership.  Other  criteria 
that  could  be  considered  would  include  'rade  point  average  in  the  three  major 
areas  and  probability  of  disenrollment . These  multiple  criteria  would  have  to 
be  combined  in  some  fashion  into  a composite  that  would  represent  the  utility 
of  assigning  a particular  person  to  a particular  major  area.  This  composite 
utility  would  be  used  in  the  payoff  matrix  to  replace  the  probability  of  group 
membership  used  in  this  study. 

As  explained  above,  the  quotas  used  for  Group  I and  Group  II  under  Strategy 
Iff*  were  assumed  to  be  equal  for  the  purposes  of  this  investigation.  Policy- 
makers at  the  Naval  Academy  would  have  to  determine  separate  quotas  for  the 
three  academic  areas  to  ensure  that  the  solution  yielded  by  the  assignment 
algorithm  would  produce  the  desired  results. 

Implementation  of  the  Findings 

Assuming  that  all  important  predictors  and  criteria  had  been  considered 
and  were  appropriately  reflected  in  the  payoff  matrix  and  assuming  the  existence 
of  an  unequivocal  institutional  policy  statement  on  quotas  for  the  three  academic 
areas.  Strategy  /M  could  be  implemented  in  various  ways.  On  the  one  extreme, 
the  assignments  produced  by  the  algorithm  could  be  used  as  the  sole  determinant 
of  an  individual's  academic  major  area.  A more  realistic  alternative  would  be 
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to  show  each  person  (1)  an  individual  vector  of  payoffs  in  the  alternative 
areas  and  (2)  the  recommended  choice,  but  allow  the  person  to  choose  from 
among  the  three  alternatives.  If  the  distribution  of  persons  in  the  academic 
areas  was  in  accordance  with  the  desired  distribution,  these  choices  could 
stand.  If  significant  discrepancies  occurred  between  the  number  of  persons 
desired  in  each  particular  area  and  the  number  choosing  a major  in  that  area, 
Academy  efforts  to  redistribute  midshipmen  could  be  focused  on  those  persons 
evidencing  the  greatest  discrepancy  between  recommended  and  actual  choice. 

In  conclusion,  it  appears  that  use  of  the  optimal  personnel  assignment 
algorithm  and  realistic  quotas,  together  with  a payoff  matrix  which  realis- 
tically portrays  the  utility  of  a particular  major  academic  area  for  a specific 
individual,  could  make  a significant  contribution  to  the  educational-vocational 
guidance  of  midshipmen. 
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APPENDIX 

OBTAINED  VERSUS  CHANCE  RESULTS  FOR 
FOUR  CLASSIFICATION  STRATEGIES 


A-0 


Obtained  and  Chance  Results  for  Four  Alternative  Strategies  for  Prediction 
of  Major  Group  Membership  for  the  Class  of  1971 


Obtained  Result* 
Por  Strategy  #1 


Claealf lcat ton  Croup 


225 

264 

171 

660 


Hit  Rate  - 48. 5X* 


Obtained  Results 
Por  Strategy  #2 


I 

II 

III 

24 

191 

10 

10 

2 36 

18 

1 

100 

70 

264 

171 


35  527 


660 


Number  Hit*  - 330 
Hit  Rate  - 50. OX* 


Obtained  Results 
Por  Strategy  #3 


1 

II 

III 

111 

95 

19 

86 

130 

48 

28 

39 

104 

225 

264 


Totals  225  264  171 


171 

660 


Number  Hits  ■ 345 
Hit  Rat*  - 52. 3X* 


Obtained  Results 
Por  Strategy  #4 


II 


IT! 
Totals 


I 

II 

III 

125 

88 

12 

101 

no 

33 

38 

46 

87 

264 

264 

132 

225 

264 

171 


Number  Hits  • 342 
Hit  Rate  - 51 .81* 


Chance  Results 
For  Strategy 


Classification  Croup 


Number  Hits  -239 
Hit  Rate  - 36. 2X 


Chance  Results 
Por  Strategy  #2 


I 

II 

111 

12 

180 

33 

14 

211 

39 

9 

137 

25 

35  528 


Number  Hits  ■ 248 
Hit  Rate  - 37.6% 


Chance  Results 
For  Strategy  #3 


I 

II 


II 

III 


I 

II 

III 

77 

90 

58 

90 

106 

68 

58 

68 

44 

225 

264 

1 70 

Numbe r Hits  • 227 
Hit  Rate  - 34. 4* 

Chan 

Por 

ce  Results 
Strategy  #4 

Claeelf lcat ion  Group 

I 11  III 

90 

90 

45 

106 

106 

53 

68 

68 

34 

264 

264 

1 32 

Number  Hits  ■ 2 30 
Hit  Rate  • 34. 8X 


•Obtained  hit  rate  Is  algnl f leant ly  greater  than  chance  hit  rete  (p'.Ol). 


Totel s 
225 

264 

172 

661 


Totals 

225 

264 

171 

660 


Totals 

225 

264 

170 

659 


Totals 

225 

265 

170 

660 
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Obtained  and  Chance  Results  for  Four  Alternative  Strategies  for  Prediction 
of  Major  Group  Membership  for  the  Class  of  1972 

Obtained  Results  Chance  Results 

For  Strategy  *1  For  Strategy  #1 


Number  Hlta  - 364  Number  Hitt  - 267 

Hit  Rate  - 56.3X*  Hit  Rate  - 41. 4* 


Obtained  Results 
For  Strategy  #2 


Totals  33  513  100  646 


Number  Hlta  - 364 
Hit  Rate  - 56. 3J* 


Chance  Results 
For  Strategy  #2 


Totals  34  513  ICO  647 


Number  Hits  • 279 
Hit  Rate  • 43. IX 


Obtained  Results  Chance  Results 

For  Strategy  #3  For  Strategy  #3 


Totals  169  307  170  646  Totals  168  307  170  645 


Number  Hits  - 365  Number  Hits  - 235 

lilt  Rate  - 56. 5X*  Hit  Rate  - 36. 4* 


Obtained  Results  Chance  Results 

For  Strategy  #4  For  Strategy  #4 


Number  Hits  • 356  Number  Hits  ■ 225 

Hit  Rate  • 55. IT*  Hit  Rate  • 34. 81 
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*0btAlned  hit  rate  is  significantly  greater  than  chance  hit  rate  (p<.01) 


Obtained  and  Chance  Results  for  Four  Alternative  Strategies  for  Prediction 
of  Major  Group  Membership  for  the  Class  of  1973 


Obtained  Results 
For  Strategy  #1 

Classification  Croup 

I II  III  Totals 


I 

9 

144 

18 

II 

5 

256 

53 

III 

1 

54 

123 

15  454  194 


Nuaber  Hits  • 388 
Hit  Rate  - 58. 5X* 


Chance  Results 
For  Strategy  11 


Classl fleet  Ion  Croup 


4 

117 

50 

7 

215 

92 

4 

122 

52 

Nuaber  Hits  - 271 
Hit  Rate  - 40. 9X 


Obtained  Results 
For  Strategy  #2 

Classification  Croup 

I 11  . III  Totals 


I 

28 

139 

4 

II 

6 

276 

32 

III 

2 

79 

97 

Totals  36  494  133  663 


Number  Hits  - 401 
Hit  Rate  - 60. 5X* 


Chance  Results 
For  Strategy  #2 


Classification  Croup 

I II  111  Totals 


171 

a. 

1 

I 

9 

127 

34 

o 

314 

1 

11 

17 

234 

63 

178 

3 

III 

10 

133 

36 

663 

Totals 

36 

494 

133 

Nuaber  Hits  - 279 
Hit  Rate  - 42. IX 


Obtained  Results 
For  Strategy  #3 

Classification  Croup 

1 11  m Totals 

92  68  11  171 


92 

68 

11 

65 

200 

49 

14 

46 

118 

Totale  171  314  178 


Nuaber  Hits  • 410 
Hit  Rate  - 61. 8X* 


Chance  Results 
For  Strategy  #3 


Classification  Croup 

1 II  III  Totals 


44 

81 

46 

81 

149 

84 

46 

84 

48 

171 

314 

178 

Number 

Hits  - 

241 

Hit  Rate  - 36. 3X 


Obtained  Results 
For  Strategy  #4 


Classification  Croup 

l II  111  Totals 


• 

122 

45 

4 

II 

110 

173 

31 

II! 

13 

47 

98 

:als 

265 

265 

133 

Nuaber  Hits  • 393 
Hit  Rate  - 59. 3X* 


Chance  Results 
For  Strategy  #4 

Classification  Croup 

I H HI  Totals 

68  f~ 68  _ 7*  170 


68 

68 

34 

126 

126 

63 

71 

71 

36 

265 

265 

133 

Nuaber 

Hits 

230 

Hit  Rat.  • 34,71 


•Obt.ln.d  hit  rat.  Is  tlRnlf Ir.ntly  graat.r  than  chanea  hit  rata  (p'.Ol). 
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Obtained  and  Chance  Results  for  Four  Alternative  Strategies  for  Prediction 
of  Major  Group  Membership  for  the  Class  of  1975-Original  Major  Groups 


i 


Obtained  Result  a 
For  Strategy  #1 


Totals  5 527  167  699 

Number  Hits  ■ 326 
Hit  Rate  - 46. 6X* 


Obtained  Results 
For  Strategy  #2 

Classl f lest  Ion  Croup 


Number  Hits  • 331 
Hit  Rate  - 47. 4X* 


Chance  Results 
For  Strategy  #1 


Totals  5 527  167  699 


Number  Hits  - 233 
Hit  Rate  - 33. 3X* 


Chance  Results 
For  Strategy  #2 


Classification  Croup 

1 II  III  Totals 


Totals  46  549  104  699 


Number  Hits  ■ 242 
Hit  Rate  - 34. 6X* 


Obtained  Results 
For  Strategy  #3 


Totals  280  255  164  699 


Number  Hits  ■ 367 
Hit  Rate  - 52. 5X* 


Chance  Results 
For  Strategy  #3 


Totals  280  255  164  699 


Number  Hits  - 243 
Hit  Rate  - 34. 8X* 


Obtained  Results 
For  Strategy  #4 


Classification  Croup 

I II  111  Totals 


Totals  280  2 79  140  699 


Number  Hits  • 366 
Hit  Rare  • 52. 4X* 


Chance  Results 
For  Strategy  #4 


Totals  280  279  140 

Number  Hits  • 247 
Hit  Rate  • 35. IX* 


"Obtained  hit  rate  la  significantly  greater  than  chance  hit  rate  (p*.01). 
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Obtained  and  Chance  Results  for  Four  Alternative  Strategies  for  Prediction 
of  Major  Group  Membership  for  the  Class  of  1975-Current  Major  Groups 


Obtained  Reaulta 
For  Strategy  #1 

Claaalf lc»r ion  Croup 


230 

382 

164 


Nuaber  Hite  • 426 
Hit  Rate  - 54. 9X* 


Obtained  Reaulta 
For  Strategy  #2 

Claaalf ication  Group 


230 

382 

164 

776 


Chance  Reaulta 
Fo-  Strategy  #1 

Claaalf ication  Croup 


Total  a 
230 

382 

164 


Hit  Rate  - 42. 3X 


Chance  Reaulta 
For  Strategy  #2 

Claaalf ication  Group 


Totala 

230 


163 

775 


Nuaber  Hite  ■ 432 
Hit  Rate  - 55. 7X* 


Nuaber  Hite  ■ 338 
Hit  Rata  - 43. 6X 


Obtained  Reaulta 
For  Strategy  #3 


II 

III 


l 

II 

III 

89 

125 

16 

108 

220 

54 

33 

37 

94 

Totala  230  382  164 


Nuaber  Hit a - 403 
Hit  Rataa  - 51,91* 


Totala 

230 

A 

1 

I 

382 

u 

o 

II 

I 

M 

164 

III 

776 

Totala 

Chance  Reaulta 
For  Strategy  #3 


I 

11 

III 

68 

113 

49 

113 

188 

81 

49 

81 

35 

230  382  165 


Nuaber  Hlta  - 291 
Hit  Rate  - 37. 5X 


Totala 

230 

382 

165 


Obtained  Reaulta 
For  Strategy  14 

Cl... Ulc.Uon  Croup 


Totala 

230 

382 


776 


Chance  Reaulta 
For  Strategy  #4 

Claaalf lcat Ion  Croup 


Totala 

230 

382 

165 

777 


IRiaber  Hite  - 389 
Hit  Rate  • 50. IX* 


Nuaber  Hlta  - 278 
Hit  Rata  - 35. 8X 


•Obtained  hit  .ate  la  aignlf leant ly  greater  than  chance  hit  rata  (p<.01) 
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